Naughty Dog established itself as a leading Playstation developer with the “Crash Bandicoot” and “Jak” video game series. With the launch of their latest franchise “Uncharted”, Naughty Dog started using motion capture for their game cinematics. Both the original installment “Drake’s Fortune” as well as the sequel “Among Thieves” have been received with open arms by critics and gamers alike. The franchise has been praised for setting a new bar for the action adventure video game genre. We recently had the opportunity to talk with Josh Scherr, cinematics animation lead for the “Uncharted” series about their use of mocap in the franchise.

MC- The Uncharted franchise was the first video game series from Naughty Dog to use motion capture. What was the motivation for incorporating Mocap into your video games?

JS- Obviously the biggest change between the Uncharted series and the Jak series was that we were moving from doing cartoony characters with stylized performances to much more realistic humans. We did plenty of keyframe animation tests early in the production of the first Uncharted; what we came to realize is that doing realistic human animation via keyframing is a very difficult and time-consuming process, and something that simply wasn't feasible given our schedules and budgets. For example, during the production of Uncharted 2, the cinematics animators were expected to do 15 seconds of finished animation per week, including facial animation. A very doable goal when working on the stylized characters of the Jak series – in fact, our quotas on those games ranged from 20-30 seconds per animator per week! – but when you're dealing with realistic humans, if you don't get the animation right, it's going to take the player completely out of the experience. We all know what people look like when they move, and we all know when it looks wrong or weird. Motion capture helps us get a good starting base, especially for all those subtleties that are really difficult to keyframe, and then we build on the performance from there.

MC- How was the decision of using motion capture for the Uncharted series first received by the animation team an how do they view it know?

JS- You know that scene in every monster movie, the one where the angry mob arrives at the mad scientist's door with pitchforks and torches? It was kind of like that. All of us at Naughty Dog come from a keyframe animation background, and the thought of using mocap was anathema to us. Ultimately, creating the best possible performance within our time constraints is the goal, and mocap is a huge help, so we all got over it. All that said, we haven't abandoned keyframing entirely, as you'll see below.

MC- What is Naughty Dog's approach to motion capture?

JS- From a technical standpoint, we're similar to a lot of other productions. We use optical motion capture, we build simple sets to match our in-game environments, and we shoot video reference for the animators. What I believe sets us apart is that we strive to get the most authentic performances we possibly can on the mocap stage, so every step in our process steers us towards that goal. First of all, our cast members do both the physical performance and the vocal performance. I've seen a lot of work done where the voice actors are recorded separately from the mocap performers, so either you've got a bunch of people pantomiming to someone else's voice, or you've got a voice actor trying to match the wild gesticulations of some dude on the mocap stage. Either way, you get this huge disconnect between the performances. Another thing that sets us apart is we record the dialogue at the same time as the physical performance, right there on the mocap stage. Having the actors perform together brings a certain spontaneity to the scenes that you just don't get when you record everyone separately in a sound booth.

We don't motion capture the faces; all of the facial animation in the game is keyframed. We also don't capture the fingers on set, we just have a library of finger poses that the animators can use. Lastly, we don't bother capturing props either. I mean, if a character's carrying a gun, you're just going to constrain the gun to their hand anyway, right?
We also spend a lot of time planning for our shoot days. If you're paying a lot of money for your stage time and your actors, the last thing you want to do is have them sitting around while you figure out how to replicate your game level using two platforms and an apple box. In addition to doing rehearsals with the actors, we also do a test run for setting up the stages, so that on the day of the shoot there's minimal down time.
Once we get the data back, we spend a lot of time finessing and massaging it. First we fix the obvious stuff like weird shoulders, gunslinger arms, IK pops, and so on. We also spend a lot of time adding keyframe animation into the mix; sometimes just as a layer on top of the mocap, other times we'll pull specific poses from the mocap as a starting point then keyframe everything in between. We'll also keyframe from scratch where necessary, usually for moments that are way too dangerous or impractical to capture on the stage, such as Drake jumping for a ledge and grabbing on with one hand.

MC- What is new in regards to motion capture in Uncharted 2?

JS- The biggest change between the two games is that we now record usable dialogue on the mocap stage. During production on the first game, we recorded the dialogue on stage as well, knowing that none of it would be usable as final audio. We simply re-recorded all the dialogue in the ADR booth later, using the stage audio as a reference. For the sequel, the stage we used was soundproofed, allowing us to use over 75% of the audio straight from the mocap stage. Otherwise, everything else we did was just a refinement of what we did on the first game.

Next Page